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Florida's Statewide Assessment Program of the State 
Department of Education has been involved in a major effort to 
develop adequate strategies for releasing and publishing assessment 
scores for schools. One dimension of this effort has been the 
development of predicted or "adjusted" scores. These scores compare a 
school to other similar schools rather than simply to the state 
average. Using complex statistical techniques^ the Florida Statewide 
Assessment Program developed and released predicted test scores for 
each- school which participated in the 1974-75 assessment of 
communication skills and mathematics in grades 3^ and 9. The 
pr^edicted scores were based on several factors which are related to 
achievement: Spanish native language^ family income^ white collar 
occupation^ college education^ and minority enrollment. This document 
is a guide to the predicted scores calculated from the 1975-76 
assessment of communication skills and mathematics in grades 3 and 6. 
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PREFACE 



The Statewide Assessment Program of the Department of Educa- 
tion has been involved in a major effort to develop adequate sti^at- 
egies for releasing and publishing assessment scores for schools. 
One dimension of this effort has been the development of predicted 
or "adjusted" scores. These scores compare a school to other 
similar schools rather thau simply to the state average. 

Using complex statistical techniques, the Statewide Assess- 
ment Program developed and released predicted test scores for each 
school which vt.^rcicipated in the 1974-75 assessment of communica- 
tion skills and mathematics in grades 3, 6, and 9. The predicted 
scores were based on several factors (primarily socio-economic) 
which are related to achievement. This document is a guide to the 
predicted scores calculated from the 1975-76 assessment of communi- 
cation skills and mathematics in grades 3 and 6. 

The Department believes that the predicted score will be a 
valuable reporting ani interpretation tgol for examining school 
achievement on statev:!de assessment. Your comments and reactions 
will be appreciated. 
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Background 



Since 1971-72, the Statewide Assessment Program has measured 
student achievement In the basic skill areas of reading, writing, 
and mathematics. Yearly, the results have Indicated to districts 
and schools what percentage of their students have achieved each of 
the staters priority objectives. These results have provided valu- 
able Information on spt^clflc strengths and weaknesses of the curric- 
ulum. 

However, the specificity and quantity of data — the two 
factors most crucial to Identifying currlc iltim strengths and weak- 
nesses — are difficult for the citizen, parent or legislator to 
understand. These audiences prefer a reporting system which sum- 
marizes the data. 

In devising a simplified method cf reporting, the Statewide 
Assessment Program attempted to avoid one drawback of common veport- 
ing methods: failure to consider the effect of outside or non- 
school factors on test scores. Many studies have documented the 
fact that test scores are influenced by non-school factors such as 
student socio-economic status. Therefore, one should look not only 
at the final test score but also at how the non-school factors in- 
fluenced that score. 

Consequently, in 1973 the Statewide Assessment Program began 
to develop a new method of reporting school scores. The method was 
first used to report the results for each school which participated 
in the 1974-75 assessment of communication skills and mathematics 
in grades 3, 6, and 9. It is an attempt to ^'predict" (adjust) a 
school's score by considering* the socio-economic characteristics 
of the students in the school. Although some complex statistical 
techniques are involved, it is basically a procedure for account- 
ing mathematically for the influence of the outside factors. 

The purpose of the predicted score is to provide a standard 
of comparison other than the state mean. Therefore, the school's 
actual performance is compared to its predicted performance level. 
The predicted score is based on the general performance trends of 
all schools with similar non-school factors. 

It is important to stress the fact that if an obsezfved 
score is higher than the predicted score the situation is not nec- 
essarily ''desirable". Both scores could be far lower than is de- 
sired. It is still the responsibility of every Florida school to 
help all of its students acquire the basic skills specified in the 
statewide objectives. This is the purpose of the objective-refer- 
enced testing and reporting conducted by the state. 



Developing the School Predicted Score 



The school predicted score Is based on available data which 
correlates highly with school test scores. A correlation means 
that two things occur together often enough across an entire popu- 
lation to allow an estimation of one if the other is known. For 
example, it is known that a relationship exists between men's 
height and weight. The correlation is not one hundred percent 
correct (there are tall, thin men), but it is stable enough to 
permit a fairly accurate estimate of weight if the man's height 
is known. 

Note that correlation does not necessarily mean causation. 
If things are related, it means only that they tend to occur to- 
gether. In the example of weight and height, a high correlation 
does not say that one causes the other. Causation may be present, 
but a correlation analysis by itself will not confirm it. 

The accuracy of the estimate or prediction can be improved 
by considering additional related factors. For example, knowing 
whether the person had a thin fraire or stocky build would help pre- 
dict his weight more accurately. 



Predictor Factors 



In trying to find the best possible predictors, the Assess- 
ment Program inveutigated many different factova (also referred to 
as "predictor variables) which were available on Department of Edu- 
cation data files. From these variables, the ones which did the 
best job of predicting school test scores were used to calculate 
the predicted ranges. The five variables are: 

1. Spanish native language , based on data gridded onto each 
student's booklet or answer sheet for the 1975-76 state- 
wide assessment; 

2. Family income , based on the percent of students receiving 
free/roduced lunch, from data In the 1975-76 Quantitative 
Report (ACC-1); 

3. Whifce cellar occupation , from the principal's estimate 
in the 19'^^^76 Quantitative Report (ACC-1) ; 

4. ColLe^e a^3ucation ^, based upon a 1975-76 survey asking 
prlncipels to estimate the percentage of students from 
famillee where either parent had at least some college 
education; and 
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5, Minority enrollment , excluding white and Spanish, based 
upon data collected in the 1975-76 Civil Rights Survey. 

These same five variables were used in the calculation of the 
1974-75 school predicted scores. (However, in 1974-75 family in- 
come was based on the principalis estimate of the- percentage of stu- 
dents with income under $3,000, instead of the percentage receiving 
free or reduced lunch.) 

Before the predictor variables were used in the prediction 
formulas, each district coordinator of accountability received a 
printout showing the values for each of these five variables for 
each School. Coordinators were asked to verify that the values 
were those originally supplied by the schools. Only after this 
verification were the 1975-76 predicted scores calculated. 

Calculating 1975-76 Scores 

The school test score, labeled in the 1975-76 reports either 
as the SCHOOL SCORE or as the OBSERVED SCORE, is based on the aver- 
age number of test items answered correctly for each subject area 
(mathematics and communication skills) and for the total test. This 
average was converted to a normal distribution in which the state 
score is 50 for each subject area and the standard deviatioa is 10. 

The purpose of the transformation is to aid interpretation 
of scores for the various subjects and grades, since different 
numbers of items were used to measure each • Without transformation, 
a score of 57 in mathematics and a score of 61 in reading are not 
comparable, as they represent 57 of 71 items versus 61 out of 102 
items . 

Because of the transformation, the following kinds of com- 
parisons can be made: 

1. If the school score is higher than the state score of 50, 
the school 16 performing above the state average in 

that area; if it is lower than 50, the school is achiev- 
ing below the state average. 

2. If the school score is higher in mathematics than in 
cor junication skills, the school is performing relative- 
ly better in mathematics. Likewise, if the score for 
third grade mathematics is higher than for sixth grade 
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mathematics, It means that third graders performed 
relatively better oa the assessment than did sixth 
graders. 

School observed scores range from a high of about 81 to a 
low of about 16 • Because of the normalization process used, about 
67% of the schools In the state have scores between 40 and 60, and 
95% have scores between 30 and 70. As a result, a school *s 1975- 
76 observed score will be higher than Its 1974-75 score only If It 
changed Its relative position In comparison to the state average. 
Even If all schools In the state had Improved In achievement of 
basic skills from 1974-75 to 1975-76, not all schools would have 
higher scores. The only schools which would have higher scores 
would be those which Improved more than the average amount of Im- 
provement statewide. 

The predicted score for the school Is found by comparing 
the school and Its characteristics (predictor values) to the 
state averages. The basic assumption Is that a school with char- 
acteristics like the state's would be expected to score at the 
state average and that, as a school 'i: characteristics differ from 
the state's, Its test scores will change correspondingly. For 
example. If a school has families with high Income and educational 
levels, we would "expect" that Its test scores would be relatively 
high. On the other hand, If the school's Income level Is lower 
than the state average we would expect that test scores would be 
lower . 

The amount by which scores will be higher or lower In each 
case has been determined by a statistical technique known as 
multiple regression. The details of the multiple regression tech- 
nique will not be given here. For further Information, contact 
your district coordinator or write to the Student Assessment Section, 
Department of Education, Tallahassee, Florida 32304. 

Reporting Predicted Scores 

Since the predictions are based on correlations between sets 
of data which are not In themselves perfectly accurate (as in the 
principals' estimates of white collar occupation and educational 
level), the predictions cannot be precise or exact. Therefore, 
each prediction takes the form of a predicted range with a predicted 
minimum and maximum score. This range is based on a confidence 
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level of ,05, which means that there is a 33% probability that the 
school's true predicted score ls-wlth':a tthe range. 

If the school's test score irA within the predicted range 
(which Includes equaling the minlau:: or the maximum) , the school 
is performing as expected <, For cjcample, if the school's score for 
mathematics is 55 and its pr^^dicnod range is 41 to 64, the school 
is within its predicted rp.rg^>. Most schools in the state are per- 
forming "as expected". 

If the school's tett score is higher than the predicted max- 
imum, the school is performing better than predicted and can be 
considered a positive outlier. If the school's test score is lower 
than the predicted minimum, the school is performing lower than 
predicted and can be considered a negative outlier. For example, 
for a predicted range of 41 to 63 for mathematics, a score of 68 
would be higher than the predicted maximum, and the school would 
be considered a positive outlier In mathematics. On the other 
hand, if the range is 41 to 63 and the school's score is 31, the 
school would be considered a negative outlier. 

An outlier school is one whose overall stxident achievement 
of the basic skills measured in the 1975-76 assessment was either 
significantly higher or lower than expected. Because of statis- 
tical errors inherent in the process, it is only possible to say 
that an outlier school is highly likely to be "unusual" in some 
way. The Department will conduct further studies of the outlier 
schools to see if it is possible to identify any instructional 
techniques which are causing good student achievement. If so, 
these techniques can be disseminated to other schools. ^ 

Reading the School Report 

The school report, called the 1975-76 PREDICTED/OBSERVED 
GRAPH (see the example on page 8), contains scores far mathematics, 
communication skills, and the total test. The school score, also 
called the OBSERVED SCORE, is the average number of items answered 
correctly in the school, transformed to a distribution with a state 
mean of 50 and a standard deviation of 10. 

There is a separate PREDICTED/OBSERVED GRAPH (see the ex- 
ample) for each grade assessed in the school. The grade and number 
of students tested are identified in the right corner. 
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The report presents two kinds of Information: (1) In the 
top part, thit observed and predicted scores for mathematics, com- 
munication skills, and total test; and (2) at the bottom, Informa- 
tion on the five factors used to calculate the predicted scores. 

In the section on observed and predicted scores, the subject 
area is identified in the center of the page. Directly below the 
subject area is the STATE SCORa for the subject, which is always 50. 

The SCHOOL SCORE for the subject area is located immediately 
below the state score. If the school score is to the left of the 
state score, the school scored below the state average. If the 
school score is to the right, the school scored above the state 
average for the subject area. 

The SCHOOL PREDICTED RANGE is represented by two scores— 
the predicted minimum and predicted maximum — which are connected 
by dotted lines. If the school score Is within the boundaries 
defined by the dotted lines, the schoo.l is performlns as predicted. 
If the school score 1b to the right of the predicted maximum, 
achievement was better than predicted. If the school score Is to 
the left of the predicted minimum, achievement was lower than 
expected. 

The bottom section cf the report provides information on 
the five non-school variables used to calculate the predicted 
scores. The listing provides th3 percentage of students In the 
bchool, the district and the state for each factor. The percent- 
age of students with Spanish native language is given by grade; 
the other factors are based upon the percentage of students in 
the school with the characteristic. The source of the data is 
listed in the right column. 

Thus, In the example, Puddlevllle Elementary had an observed 
school score of 53 in mathematics, which is slightly above the 
state score of 50 and within the school *s predicted range of 35 to 
60. Therefore, Puddlevllle is performing as predicted in mathe- 
matics. The school *s score of 66 in communication skills is sub- 
stantially above the state score of 50 and la also above the 
predicted range of 36 to 59. Thus, Puddlevllle is a positive 
outlier for communication skills and is also a positive outlier 
on total test, since its score of 60 is above the predicted range 
of 36-59. 
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At the bottom of the report are the values of the predictor 
variables which were the basis for the predicted scores. They show 
that for sixth grade 1% of the students in the school have Spanish 
as their native language. Thirty-eight percent of the students are 
considered low income (free/reduced lunch) , 25% come from families 
employed in white collar occupation, 14% come from families with 
some college education, and 20% are minority students. 



District Scores 



District scores have been calculated as part of the 1975-76 
predicted score process. These distr.-^^zt scores are weighted aver- 
ages, based on the population of each school. They are printed on 
the 1975-76 PREDICTED/OBSERVED LISTING (example, page 10). 

Ca utions in Interpreting Predicted Scores 

There are a number of special limitations which apply to the 
interpretation of the 1975-76 predicted scores. First, the results 
are based only on student achievement of basic skills in reading, 
writing, and mathematics. Conclusions about school effectiveness 
should be limited to these subject areas, rather than to the entire 
curriculum. 

Second, the scores are obtained through a process which 
forced a non-normal distribution into a normal shape. Because 
Statewide Assessment is a measure of minimal skills, it does not 
discrlDiinate well among the top students, and large percentages of 
them achieve high scores on the test. This mastery curve is appro- 
priate for an cjjective-ref erenced test of basic skills. However, 
forcing the schools' averages into a normal distribution implies 
more separation between the top schools than actually exists. 

Third, the predictor variables are based on estimates, so 
the exact status of any one school may be in error. However, the 
amount of error is not large enough to move a school from the 
positive outlier area to the negative outlier category, or vice 
versa. 

Fourth, the identification of a school as a positive or 
negative outlier is not a sure selection of the most or least 
effective schools in the state. Much further information must be 
gathered throtigh on-site visitations to determine whether or not 
the initial classification of the school was Justified. 
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Using the Predicted Scores In Reports 



Good dissemination practices Include release of susmary test 
results to various audiences such as parents, the school board, and 
the news media. The school observed and predicted range scores pro- 
vide simplified data which can be used for ..his purpose. The data 
can be reported either In narrative format or In a combination of 
narrative and tables. 

EXAMPLE 1. —NARRATIVE 

During 1975-76, all etudente in grades 3 and 6 were aBBeeeed 
or. statewide obo'eotivee in reading, writing, and mathematics. The 
results indicate that^.., (mention key strengths and weaknesses 
from the local district analysis of performance on each objective) . 

The complete scores are available in the school office, but, 
in summary J they show that, . . 

1. Third grade math: The school score of 5C is above its 
predicted range cf 11 to 45 and the same as the state 
score of 50. This means our school is performing better 
than other similar schools. 

2. Third grade communication skills: The school score of 
40 is within its predicted range of 17 to 44, but 
slightly below the state score of 50. Therefore, the 
scJiool is performing as well as other simitar schools, 
h%it its average is less than the state average. 

EXAMPLE 2. —TABLE 

Use the same lead-in paragraph as In example 1. Use a table 
similar to the one below: 



STATE SCHOOL 
SCORE SCORE 



SCHOOL 
PREDICTED RANGE 



INTERPRETATION 



3rd gr. Math 



SO 



50 



10^45 



School scored above 
predicted, and same 
as state. 
School scored as 
predicted, but below 
state average. 
School scored as 
predicted, but below 
state average. 



Zrd gr. Comm. 



50 



40 



17^44 



Zrd gr. Total 



50 



43 



17^44 



